Optimizing Parallel Applications for Wide-Area Clusters

Recent developments in networking technology cause a growing interest in connecting local-area clusters of workstations over wide-area links, creating multilevel clusters. Often, latency and bandwidth differences between local-area and wide-area network links are two orders of magnitude or more. With such a large difference, one would expect only very coarse grain applications to achieve good performance. Hence, most meta computing endeavors are targeted at job-level parallelism. To test this intuition, we have analyzed the behavior of several existing mediumgrain applications on a wide-area multicluster. We find that, if the programs are optimized to take the multilevel network structure into account, most obtain high performance. The optimizations we used reduce intercluster traffic and hide intercluster latency, and substantially improve performance on wide-area multiclusters. As a result, the range of applications suited for a meta computer is larger than previously assumed.